Remote handling is the remote control by a human operator of a manipulator arm, a mobile machine or any other articulated powered device. In the remainder of the description, a device of this type is denoted by the term “manipulator”. In a complete remote handling system, the human operator interacts remotely with the manipulator by means of an articulated master arm whose movements are captured, transmitted and reproduced by the manipulator. To perform this manipulation in real time, the operator requires continuous high-quality visual feedback on the working environment which enables him to execute the current task accurately and in a completely safe way. This visual viewpoint is delivered by a camera mounted on the system which includes the manipulator. This camera can be positioned on the end of the manipulator arm if a precise local viewpoint of the operating area is desired. However, this positioning has the drawback of limiting the operator's field of view and cannot give him an overall view of his working environment. To ensure that the operation takes place in a completely safe way, it is often necessary to have a second viewpoint which shows the manipulator in the overall scene. This overall viewpoint is generally obtained by means of a motorized camera of the PTZ (“Pan Tilt Zoom”) type.
One of the problems to be resolved in a remote handling system is that of providing the operator at all times with a relevant viewpoint onto an area of interest. For some applications, the environment in which the manipulator moves is entirely known and is under control and free of obstacles. In this case, solutions using one or more fixed cameras in the environment may be adequate. However, if a moving point is to be followed, this point may be located outside the field of view, or may be concealed by the presence of objects in the environment or even by the manipulator itself. In such cases the image delivered to the operator is unusable. One way of overcoming this problem is to use a motorized camera, such as a PTZ (Pan Tilt Zoom) camera. A second operator, capable of controlling the movement of the motorized camera, can then update the line of sight of the camera manually. This method can also be automated by making the camera viewpoint dependent on the area of interest, whose position is known in advance. When the manipulator makes a movement, the position of the point of interest is converted to a viewpoint set point for the camera. The viewpoint of the camera is then updated as a function of this set point. This method produces good results when the line of sight of the camera is clear.
The problem to be resolved by the present invention is therefore that of providing an operator, who is manipulating an object remotely, with a real-time relevant image of the scene in which he is working. A known solution to this problem is that of placing a camera on a second robotized machine whose position is updated by a second operator. However, this solution is unsatisfactory, as it requires human intervention. The production of this image, and indirectly the adjustment of the camera viewpoint, must be carried out automatically, without including a second operator responsible for continuously correcting the viewpoint to obtain a clear field of view onto the object to be manipulated.
Various known prior art methods have been developed to overcome the aforesaid problem.
The most easily implemented solutions for giving the operator a relevant viewpoint are of the type described in International Patent Application WO 2007/067167. The target is located by means of a transmitter positioned on it and a receiver coupled to the motorized camera. The transmitter transmits the position of the target in real time to the receiver, which communicates this information to the system responsible for updating the orientation of the camera to ensure that its line of sight is always aimed toward the target. This device is simple, but cannot provide a relevant image for the operator if masking obstacles are present between the camera and the target. This type of solution is unsuitable, as the environment is not taken into consideration.
A different solution is developed in the article “Dynamic Sensor Planning in Visual Servoing”, by Eric March and and Greg D. Hager, ICRA '98. In this solution, a viewpoint free of obstacles is obtained by means of a camera mounted on a robotized arm, with the aim of enabling the target object to be grasped subsequently without difficulty. The operations described in this document use the known methods of visual servoing. These methods are based on the definition of cost functions which are to be minimized. The function cost is declared to be minimal when the target objects are located in the centre of the image and the masking objects are at the periphery of the image or outside it. The aim is then to adapt the position of the robotized machine continuously so that the projection of the target object on the image and therefore its representative surface in the image is always as large as possible in proportion with respect to the other objects in the scene.
Another known prior art solution uses what are known as model-referenced path planning methods. Based on a geometrical model of the scene, these methods seek a means for connecting a point A to a point B while avoiding the obstacles. An example of an embodiment of this method is described in U.S. Pat. No. 5,808,887. By applying this method to the problem of remote handling, it is possible to determine the new camera position which is closest to its starting point while allowing a straight line to be drawn without any interference caused by an obstacle between the target and the new camera position.
A considerable drawback of visual servoing and path planning methods is that both methods are very demanding in terms of computing power, which limits their use for real time applications. Furthermore, the major problem of visual servoing methods is that they are dependent on image quality and require a high level of contrast between the various objects in the scene if an effective solution is to be found. They are also highly sensitive to the effects of light, causing problems in a context in which the camera support is movable and the incidence of light on the objects in the scene is variable.